1,946 research outputs found
Efficient end-to-end learning for quantizable representations
Embedding representation learning via neural networks is at the core
foundation of modern similarity based search. While much effort has been put in
developing algorithms for learning binary hamming code representations for
search efficiency, this still requires a linear scan of the entire dataset per
each query and trades off the search accuracy through binarization. To this
end, we consider the problem of directly learning a quantizable embedding
representation and the sparse binary hash code end-to-end which can be used to
construct an efficient hash table not only providing significant search
reduction in the number of data but also achieving the state of the art search
accuracy outperforming previous state of the art deep metric learning methods.
We also show that finding the optimal sparse binary hash code in a mini-batch
can be computed exactly in polynomial time by solving a minimum cost flow
problem. Our results on Cifar-100 and on ImageNet datasets show the state of
the art search accuracy in precision@k and NMI metrics while providing up to
98X and 478X search speedup respectively over exhaustive linear search. The
source code is available at
https://github.com/maestrojeong/Deep-Hash-Table-ICML18Comment: Accepted and to appear at ICML 2018. Camera ready versio
Parsimonious Black-Box Adversarial Attacks via Efficient Combinatorial Optimization
Solving for adversarial examples with projected gradient descent has been
demonstrated to be highly effective in fooling the neural network based
classifiers. However, in the black-box setting, the attacker is limited only to
the query access to the network and solving for a successful adversarial
example becomes much more difficult. To this end, recent methods aim at
estimating the true gradient signal based on the input queries but at the cost
of excessive queries. We propose an efficient discrete surrogate to the
optimization problem which does not require estimating the gradient and
consequently becomes free of the first order update hyperparameters to tune.
Our experiments on Cifar-10 and ImageNet show the state of the art black-box
attack performance with significant reduction in the required queries compared
to a number of recently proposed methods. The source code is available at
https://github.com/snu-mllab/parsimonious-blackbox-attack.Comment: Accepted and to appear at ICML 201
Deep Metric Learning via Facility Location
Learning the representation and the similarity metric in an end-to-end
fashion with deep networks have demonstrated outstanding results for clustering
and retrieval. However, these recent approaches still suffer from the
performance degradation stemming from the local metric training procedure which
is unaware of the global structure of the embedding space.
We propose a global metric learning scheme for optimizing the deep metric
embedding with the learnable clustering function and the clustering metric
(NMI) in a novel structured prediction framework.
Our experiments on CUB200-2011, Cars196, and Stanford online products
datasets show state of the art performance both on the clustering and retrieval
tasks measured in the NMI and Recall@K evaluation metrics.Comment: Submission accepted at CVPR 201
Deep Metric Learning via Lifted Structured Feature Embedding
Learning the distance metric between pairs of examples is of great importance
for learning and visual recognition. With the remarkable success from the state
of the art convolutional neural networks, recent works have shown promising
results on discriminatively training the networks to learn semantic feature
embeddings where similar examples are mapped close to each other and dissimilar
examples are mapped farther apart. In this paper, we describe an algorithm for
taking full advantage of the training batches in the neural network training by
lifting the vector of pairwise distances within the batch to the matrix of
pairwise distances. This step enables the algorithm to learn the state of the
art feature embedding by optimizing a novel structured prediction objective on
the lifted problem. Additionally, we collected Online Products dataset: 120k
images of 23k classes of online products for metric learning. Our experiments
on the CUB-200-2011, CARS196, and Online Products datasets demonstrate
significant improvement over existing deep feature embedding methods on all
experimented embedding sizes with the GoogLeNet network.Comment: 11 page
Neural Relation Graph: A Unified Framework for Identifying Label Noise and Outlier Data
Diagnosing and cleaning data is a crucial step for building robust machine
learning systems. However, identifying problems within large-scale datasets
with real-world distributions is challenging due to the presence of complex
issues such as label errors, under-representation, and outliers. In this paper,
we propose a unified approach for identifying the problematic data by utilizing
a largely ignored source of information: a relational structure of data in the
feature-embedded space. To this end, we present scalable and effective
algorithms for detecting label errors and outlier data based on the relational
graph structure of data. We further introduce a visualization tool that
provides contextual information of a data point in the feature-embedded space,
serving as an effective tool for interactively diagnosing data. We evaluate the
label error and outlier/out-of-distribution (OOD) detection performances of our
approach on the large-scale image, speech, and language domain tasks, including
ImageNet, ESC-50, and MNLI. Our approach achieves state-of-the-art detection
performance on all tasks considered and demonstrates its effectiveness in
debugging large-scale real-world datasets across various domains.Comment: preprin
EMI: Exploration with Mutual Information
Reinforcement learning algorithms struggle when the reward signal is very
sparse. In these cases, naive random exploration methods essentially rely on a
random walk to stumble onto a rewarding state. Recent works utilize intrinsic
motivation to guide the exploration via generative models, predictive forward
models, or discriminative modeling of novelty. We propose EMI, which is an
exploration method that constructs embedding representation of states and
actions that does not rely on generative decoding of the full observation but
extracts predictive signals that can be used to guide exploration based on
forward prediction in the representation space. Our experiments show
competitive results on challenging locomotion tasks with continuous control and
on image-based exploration tasks with discrete actions on Atari. The source
code is available at https://github.com/snu-mllab/EMI .Comment: Accepted and to appear at ICML 201
A Suspended Nanogap Formed by Field-Induced Atomically Sharp Tips
A sub-nanometer scale suspended gap (nanogap) defined by electric field-induced atomically sharp metallic tips is presented. A strong local electric field (\u3e109 V=m) across micro/nanomachined tips facing each other causes the metal ion migration in the form of dendrite-like growth at the cathode. The nanogap is fully isolated from the substrate eliminating growth mechanisms that involve substrate interactions. The proposed mechanism of ion transportation is verified using real-time imaging of the metal ion transportation using an in situ biasing in transmission electron microscope (TEM). The configuration of the micro/nanomachined suspended tips allows nanostructure growth of a wide variety of materials including metals, metal-oxides, and polymers. VC 2012 American Institute of Physics
- โฆ